The present invention relates to motion picture encoding systems, and more particularly, to an improvement of encoding motion video signal in video signal transmission systems such as television telephone systems.
In a commonly termed video signal transmission system such as a television conference or television telephone system, in which a video signal is transmitted as a motion video signal to remote places, the video signal is encoded by utilizing its line correlation or inter-frame correlation. In this way, the efficiency of transmission of useful information is increased, thus making effective use of the transmission line.
FIG. 1 illustrates, at the top portion, an example of intra-frame encoding utilizing the line correlation of a video signal for transmitting the contents of picture data PC1, PC2, . . . at respective instants t.sub.1, t.sub.2, . . . In this case, the picture data are processed for transmission by unit-dimensional encoding in the same scanning line.
FIG. 1 also illustrates, at the bottom portion, an example of inter-frame encoding utilizing inter-frame correlation of video signal. In this case, the pixel data differences between adjacent frame picture data PC1 and PC2, PC2 and PC3, . . . are obtained as picture data PC12, PC23, . . . to be transmitted, thereby improving the data compression factor.
Thus, this video signal transmission system can send out high efficiency encoded digital data to the transmission line, the data quantity of the digital data being far smaller compared to the case of transmitting all the picture data of the picture frames PC1, PC2, . . .
FIG. 2 shows a motion picture encoding system 1. Here, an input video signal VD is fed to a pre-processing circuit 2 for conversion into luminance signal and color difference signal and then conversion in an analog-to-digital converter (not shown) into an 8-bit digital signal, which is output as input picture data S1.
The picture data which are sent out consecutively as the input picture data S1, are extracted from frame picture data FRM in a manner as shown in FIG. 3.
As shown in FIG. 3, the frame picture data FRM of one picture frame is divided half horizontal by two set of 6 vertically grouped blocks GOB. Each grouped block GOB consists of 11 (horizontal) by 3 (vertical) macro-blocks MB. Each macro-block includes color difference data C.sub.b and C.sub.r which are constituted by color difference data corresponding to all the pixel data of luminance data Y.sub.1 to Y.sub.4 of 8 by 8 pixels.
In the grouped block GOB, the picture data are arranged such that they are continuous in units of macro-blocks MB. In the macro-blocks MB, picture data are continuous in units of very small blocks in the order of the raster scan.
In the macro-blocks MB, picture data (Y.sub.1 to Y.sub.4) of 16 by 16 pixels continuous in the horizontal and vertical scanning directions with respect to the luminance signal constitute a single unit as a block of data. On the other hand, the two corresponding color difference signals are processed for data quantity reduction and then time base multiplex processed to assign each of the very small blocks C.sub.r and C.sub.b to the 16 to 16 by pixel data.
A difference data generation circuit 3 receives the input picture data Sl and also the preceding frame picture data S2 of the preceding frame stored in a preceding frame memory 4 and thereupon obtains the difference between the data S1 and S2 to generate inter-frame encoded data (which is hereinafter referred to as inter-frame encoding mode). The inter-frame encoded data is output as difference data S3 to be coupled through a switching circuit 5 to a discrete cosine transformation (DCT) circuit 6 and also to a switching control circuit 7.
The switching circuit 5 is controlled by a control signal S4 which is output from the switching control circuit 7. If it is found that data can be transmitted in less data quantity by intra-frame encoding the data, the circuit 5 outputs the input picture data S1. On the other hand, if it is found that data can be transmitted in less data quantity by inter-frame encoding the data, the difference data S3 is output.
The DCT circuit 6 discrete cosine transforms the input picture S1 or difference data S3 in units of very small blocks by utilizing the two-dimensional correlation of video signal and outputs resultant discrete cosine transformed data S5 to a quantization circuit 8.
The quantization circuit 8 quantizes the transformed data S5 in a quantization step size which is determined for each grouped block GOB. Quantized data S6 that is obtained at the output terminal of the quantization circuit 8 is fed to a variable length coding (VLC) circuit 9 and also to an inverse quantization circuit 12.
The VLC circuit 9 variable length encodes the quantized data S6 and feeds the result as transmission data S7 to a transmission buffer memory 10.
The transmission buffer memory 10 stores the transmission data S7 and outputs it as output data S8 at a predetermined timing to a transmission line 11. Also, the memory 10 feeds back a quantization control signal S9 in units of grouped blocks GOB to the quantization circuit 8 according to the quantity of residual data in the memory 10 for quantization step size control.
In this way, the transmission buffer memory 10 controls the quantity of data generated as the output data S8 to retain an adequate residual quantity (i.e., a quantity not to cause overflow or underflow) of data in it.
When the quantity of residual data in the transmission buffer memory 10 is increased up to a permissible upper limit, the transmission buffer memory 10 increases the quantization step size, i.e., quantization step size STPS (in FIG. 4) of the quantization circuit 8 according to the quantization control signal S9, thus reducing the data quantity of the quantized data S6.
When the quantity of residual data in the transmission buffer memory 10 is conversely reduced down to a permissible lower limit, the transmission buffer memory 10 reduces the quantization step size STPS of the quantization circuit 8 to increase the data quantity of the quantized data S6.
The inverse quantization circuit 12 inverse quantizes the quantized data S6 output from the quantization circuit 8 to a typical value, thus obtaining inverse quantized data S10, and decodes the quantized data S6 into inverse quantized data equivalent to the transformed data S5 before quantizing in the quantization circuit 8. The inverse quantized data S10 is fed to an inverse discrete cosine transformation (IDCT) circuit 13.
The IDCT circuit 13 processes the inverse quantized data S10, obtained by the decoding in the inverse quantization circuit 12, in an inverse manner to the process in the DCT circuit 6, to obtain decoded picture data S11, which is output to a preceding frame data generation circuit 14 and also to a switching circuit 15.
Thus, the output data S8, which is sent out to the transmission line 11 and reproduced on the receiving side, can be decoded on the transmitting side as well, i.e., in the IDCT 13, to restore the input picture data S1 or difference data S3 before transforming in the DCT circuit 6.
More specifically, when the video signal VD is intra-frame encoded for transmission, the IDCT 13 restores the input picture data S1, while it restores the difference data S3 in case of inter-frame encoding the video signal VD for transmission.
The frame data generation circuit 14 adds together with the preceding frame data S2 fed back from the frame memory 4 and the decoded picture data S11 to restore the preceding frame data that has been output as the output data S8. The restored data is fed through the switching circuit 15 to the preceding frame memory 4. In this way, pictures which are transmitted to the receiving side are consecutively restored and stored in the preceding frame memory 4.
The switching circuit 15 is switched under control of the control signal S4 which is delayed by the delay circuit 16 for a period of time required from the discrete cosine conversion of the video signal VD until the inverse discrete cosine conversion.
In the motion picture encoding system having the above construction, it is presently the practice to alter the encoding system for the control of the quantity (i.e., bit quantity) of generated information. However, the control does not depend on the content or character of the picture data. There have been some trials of encoding systems, the control of which depend on the picture. However, no present system can flexibly deal with various different kinds of pictures, and in effect it has been impossible to enhance the subjective picture quality.
Further, in such motion picture encoding systems, it is determined for each frame in accordance with a certain pattern whether intra-frame encoding or inter-frame encoding based on forward, rearward and bidirectional prediction is to be done. Therefore, when the statistical characteristic of the motion picture is changed, an optimum encoding manner can not be selected. Consequently, it is impossible to expect high encoding efficiency.